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Detailed Action 



Preliminary Amendments 



1 . The preliminary amendment filed May 15, 2001 has been made of record. The 
Claims have been amended accordingly. The amendments to the Title, Specification and 
Abstract have been acknowledged. These amendments do not introduce new matter. 



Objections 

2. The drawings are objected to because of the following. The axes depicted in Fig. 
2 are too light. A proposed drawing correction or corrected drawings are required in reply 
to the Office action to avoid abandonment of the application. The objection to the 
drawings will not be held in abeyance. 



Objections 

3. The disclosure is objected to because of the following informalities: 

a. Referring to the Applicant's preliminary amendment, the paragraphs on 



Drawings 



Specification 



page 4 (line 10) to page 5 (line 15) do not expound any further on Figs 1- 
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2 than what was given on page 4, lines 1-9. The Applicant would be 
better suited placing these paragraphs in either the Summary or the 
Detailed Description, and maintain lines 1-9 on page 4 as the Brief 
Description of the Drawings. 

b. In the amended Abstract (line 7), the Applicant states, "...matching a 
current pixel image with a pixel of another image...". Since pixel images 
are not typically matched to individual pixels, it will be assumed that the 
Applicant intended this phrase to be "...matching a current image pixel 
with a pixel of another image...". The Abstract should be changed to 
reflect this. 

c. In the amended Abstract (line 9), the word othrr should be changed to 



d. Throughout the Applicant's specification, the Applicant frequently uses 
the phrase "consisting in" (e.g. Preliminary Amendment page 3, lines 15). 
Generally, when using the word consisting in conjunction with the word in, 
the word consisting takes on the meaning residing. While this is not an 
entirely incorrect usage of the word consist, within the context of claims 2- 
3 - e.g. the process of navigation in a 3D scene resides or lies in creating 
images, etc. - such usage is atypical. Typically, one would use the 
phrase "consisting of in such contexts - e.g. the process of navigation in 
a 3D scene consists of, or comprises, creating images, etc. The Applicant 
should change all improper instances of the phrase "consisting in" to 
"consisting of to ensure that the disclosure adheres to at least a marginal 
degree of grammatical rigor. 

e. The faulty grammar of the phrase "...a less good resolution than a 



other. 
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distribution ..." (page 5, lines 19 of the original Disclosure) should be 
corrected. 

f. On page 6, lines 10 of the original disclosure, "of this current image" 
should be changed to "of the current image". 
4. Appropriate correction is required. 



Claims 

Objections 

5. Claim 2-3 are objected to because of the following informalities. On line 6 of 
Claim 2, the text "(6)" serves no purpose and should be removed. 

6. Claims 2-3 use the phrase "in that" (Claim 2, line 5 and Claim 3, line 3). In both 
instances, the phrase seems to be extraneous, and its removal does not substantially 
alter the meaning of the claims or their consistency with respect to the Applicant's 
disclosure. If "that" indeed refers to some previous step of the respective claims (i.e. the 
calculation of weights in claim 2 and the assignation of relevance in claim 3), then the 
Applicant should construct the language of claims 2-3 to better reflect this. Appropriate 
correction is required. 

7. Claim 4 is objected to because of the following informality. The language of claim 
4 is unnecessarily awkward. Specifically, the phraseology of the following excerpt, "... 
and ...the other images of the sequence for which the matching of the pixels of the 
current image is performed are the images of its list", is so poor that it obfuscates the 
feature being claimed. A more intelligible alternative to this language could be 
(assuming the aforementioned phrase has been properly deciphered), "... and ... the 
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images belonging to the list associated with the current image are images of the 
sequence that are to be used in the matching of the current frame's pixels". Appropriate 
correction is required. 

8. Claim 1 1 is objected because of the following informality. Claim 1 1 contains the 
phrase "consisting in". Following the discussion above, this should be changed to 
"consisting of. Appropriate correction is required. 

9. The Applicant claims his/her invention as a process. Although not improper, 
simply replacing the word process with method {process and method are synonymous 
according to U.S.C. § 100(b)) would make the claim language more consistent with the 
Applicant's disclosure and Title of Invention. 



Rejections Under 35 U.S.C. §112(2) 

10. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and 
distinctly claiming the subject matter which the applicant regards as his invention. 

1 1 . Claims 2 and 1 1 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

12. The following is in regard to Claim 2. In claim 2, the Applicant puts forth a step 
involving the "new selection of the pixels is performed depending on the resolution and 
weight values assigned to the pixels". This statement implies that there is a selection of 
pixels (i.e. a new selection) following a previous selection, presumably the "selection of a 
pixel of the current image depending on its resolution and on that of the pixels of other 
images of the sequence matched with this pixel" put forth in claim 1. This, however, is 
not consistent with the Applicant's disclosure (see, for example, Fig. 1). According to the 
disclosure, pixel selection is based on the resolution and the weight (Applicant's original 
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specification page 7, lines 25-36) and is made subsequent to the calculation of weights. 
Therefore, there is no prior pixel selection based on the resolution, followed by another 
(new) pixel selection based on the calculated weights and resolution. The selection of 
pixels put forth in Claim 2 will be treated, henceforth, simply as a "selection of the pixels 
is performed depending on the resolution and weight values assigned to the pixels", 
without that selection having the distinction of being new. 

13. The following is in regard to Claim 11. Claim 1 1 recites "...creating images as a 
function of the movement of the viewpoint, wherein the images are created on the basis 
of the process for constructing the 3D model according to Claim 1". The Applicant's 
invention according to claim 1 has no step or any other means to create images. Rather, 
the product of the method of claim 1 is a 3D model derived from a sequence of images. 
Claim 1 1 will be interpreted, henceforth, as follows: 

Method of navigation in a 3D scene consisting of creating images as a function of the movement of 
the viewpoint, wherein the images are of different viewpoints of the 3D model derived according to 
Claim 1 . 

Rejections Under 35 U.S.C. $ 102(e) 

14. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United 
States only if the international application designated the United States and was published under 
Article 21(2) of such treaty in the English language. 



15. Claims 1, 9 and 11 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Davidson et al. 
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16. The following is in regard to Claim 1. Davidson et al. disclose a method for 
constructing a 3D scene model by analyzing image sequences, each image 
corresponding to a viewpoint defined by its position and its orientation (see Davidson et 
al. Fig. 2 and Abstract), wherein it comprises the following steps: 

(1 .a.) Calculation, for an image, of a depth map corresponding to the depth, in 
3D space, of the pixels of the image. See Davidson et al. Fig. 3, step S10 
and note depth is inherent to 3D data. 
(1 .b.) Calculation, for an image, of a resolution map (i.e. set of calculated inter- 
point distances (SHIFT) - Davidson et al. Figs. 43-45b) corresponding to 
the 3D resolution of the pixels of the image, from the depth map (i.e. 3D 
data - Davidson et al. Fig. 3, step S10). See Davidson et al. column 43, 
lines 3-4, column 44, lines 1-40. 3D resolution is taken to be the inter- 
point distance, which is a measure of pixel dispersion. This definition is 
essentially the same as the Applicant's definition of 3D resolution. 
(1 .c.) Matching of a pixel of a current image with a pixel of another image of the 
sequence (e.g. Davidson et al. Fig. 3, step S8, column 7, lines 39-45 and 
column 10, lines 53-61). 

1 . The matching is achieved by projecting the pixel of the current image 
onto the other image (e.g. Davidson et al. column 18, lines 34-59 
and/or column 33, lines 38-53 and/or column 41 , lines 40-55 and 
column 42, lines 31-54). 

2. The matching pixels relate to one and the same point of the 3D scene 
(e.g. Davidson et al. Figs. 29a). 

(1 .d.) Selection of a pixel of the current image depending on its resolution and 
on that of the pixels of other images of the sequence matched with this 
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pixel. Notice steps 558-560 of Davidson et al. Fig. 44a. Here, discarding 
points of excessive inter-point distance (i.e. low 3D resolution), results in 
the selection of a set of points, and correspondingly a set of pixels, used 
in the subsequent construction. 
(1 .e) Construction of the 3D model from the selected pixels (Davidson et al. 
Fig. 3, steps S12-S14). 

It has thus been shown that the 3D model construction method of Davidson et al. 

sufficiently conforms to the method proposed by the Applicant in claim 1 . Therefore, the 

teachings of Davidson et al. anticipate the method of claim 1. 

17. The following is in regard to Claim 9. As shown above, Davidson et al. disclose a 
3D model construction method that sufficiently conforms to the method of claim 1. In the 
method of Davidson et al., the pixel of the other image is the pixel closest to the 
projection point on this other image. See, for example, Davidson et al. Fig. 35. The 3D 
model construction method of Davidson et al. thus sufficiently conforms to the method 
proposed by the Applicant in claim 9. Therefore, the teachings of Davidson et al. 
anticipate the method of claim 9. 

1 8. The following is in regard to Claim 1 1 As shown above, Davidson et al. disclose 
a 3D model construction method that sufficiently conforms to the method of claim 1 . 
Davidson et al. suggest the usage of the derived model in virtual reality applications (e.g. 
ones that employ VRML - Davidson et al. column 6, lines 3-5). The following aspects of 
claim 1 1 are inherent to virtual reality applications: 

(1 1 .a'.) Creating images as a function of the movement of the viewpoint. 
(1 1 .b'.) These images are of different viewpoints of a 3D model. 
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Therefore, a virtual-reality application utilizing a model(s) obtained according to the 
method of Davidson et al., would constitute a method of navigation in a 3D scene 
consisting of: 

(1 1 .a.) Creating images as a function of the movement of the viewpoint. 
(1 1 .b.) These images are of different viewpoints of the 3D model(s) derived 
according to Claim 1. 
This is clearly in accordance with the method proposed in claim 1 1 . 

Rejections Under 35 U.S.C. § 103(a) 

1 9. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

20. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Davidson 
et al., in view of Azarbayejani et al. (U.S. Patent 5,51 1,153). 

21 . The following is in regard to Claim 2. As shown above, Davidson et al. disclose a 
3D model construction method that sufficiently conforms to the method of claim 1 . 
Clearly, the points selected according to the method of Davidson et al. constitute one or 
more regions. Furthermore, as stated above, the method of Davidson et al. selects 
pixels depending on the resolution (see the discussion relating to step (1.d) above). 
However, Davidson et al. do not expressly show or suggest: 

(2.a.) Calculation and allocation of weights to the pixels of the image depending 
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on: 

1 . whether or not they belong to the regions 

2. on the geometrical characteristics of the regions to which they belong 
in the image. 

(2.b.) A selection of the pixels performed depending on the weight values 
assigned to the pixels. 

22. Azarbayejani et al. 1 , on the other hand, disclose a method for constructing a 3D 
scene model by analyzing image sequences, wherein: 

(2.a'.) Weights (e.g. i/(i + z c /?) in Azarbayejani et al. equation (2) or a in 
Azarbayejani et al. equation (3)) are calculated and allocated to the 
pixels of the image. See Azarbayejani et al. equations (2) and (3), for 
example. The weights a,... a N represent the geometric characteristics or 
structure of an image feature (i.e. region) to which the pixels belong 
(Azarbayejani et al. column 6, lines 5-13) and are indicative of whether or 
not they belong to the feature (Azarbayejani et al. column 5, lines 54-60). 

(2.b\) A selection of the pixels performed depending on the weight values 
assigned to the pixels (Azarbayejani et al. column 10, lines 52-61). 

23. The teachings of Davidson et al. and Azarbayejani et al. are combinable because 
they are analogous art. Specifically, the teachings of both are directed to 3D model 
generation from image sequences. Therefore, it would have been obvious to one of 
ordinary skill in the art, at the time of the applicant's claimed invention, to perform feature 
tracking (or matching, using Davidson et al.'s nomenclature) in the method of Davidson 



1 Note that the various equations of Azarbayejani et al. will be referred to as (1 ), (2), (3) and so 
on, beginning with the equation listed in Azarbayejani et al. column 2, line 40. 
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et al. according to steps (2.a\)-(2.b\) of Azarbayejani et al. One would have been 
motivated to do so because of the accuracy of Azarbayejani et al.'s method. Combining 
the teachings of Davidson et al. and Azarbayejani et al. yields a method, in accordance 
with claim 1 , such that: 

(2.a.) Calculation and allocation of weights to the pixels of the image depending 
on: 

1 . whether or not they belong to the regions 

2. on the geometrical characteristics of the regions to which they belong 
in the image. 

(2.b.) A selection of the pixels performed depending on the resolution and 
weight values assigned to the pixels. 
Therefore, such a method would conform to the method proposed in claim 2. 

24. Claims 4-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Davidson et al., in view of McAllister et al. ("Real-Time Rendering Techniques of Real 
World Environments"). 

25. The following is in regard to Claim 4. As shown above, Davidson et al. disclose a 
3D model construction method that sufficiently conforms to the method of claim 1 . 
Davidson et al., however, do not expressly show or suggest a partitioning of the images 
of the sequence characterized by: 

(4.a.) Identifying, for a current image, the images whose corresponding 

viewpoints have an observation field possessing an intersection with the 
observation field relating to the current image. 

(4.b.) Forming a list of images associated therewith. 
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(4.c.) The list contains images of the sequence that will be used in the 
matching of the pixels of the current image. 

26. McAllister et al. disclose an image-based modeling method that generates a 3D 
model of an object from a multitude of 2D images (McAllister et al. Abstract). The 
method entails a partitioning of the images (i.e. into tiles) of the sequence (McAllister et 
al. Section 4.5, paragraph 1) that includes the following: 

(4.b.) Forming a list of images associated therewith (McAllister et al. Section 
4.6, paragraph 2). 

(4.c.) The list contains images of the sequence that will be used in the 

matching of the pixels of the current image (McAllister et al. Section 4.6, 
paragraph 2, in particular, lines 4-7). 

Note that: 

(4.a.) Identifying, for a current image, the images whose corresponding 

viewpoints have an observation field (i.e. view frustum - McAllister et al. 
Section 4.5 paragraph 1, line 4) possessing an intersection with the 
observation field relating to the current image, 
is inherent to the view-frustum culling of McAllister's et al.'s method. Those view- 
frustums that are culled correspond to images that sample surfaces which are disjoint 
from the surfaces imaged by the retained view-frustums. One can conclude from this 
that the retained and culled view-frustums are divergent. Thus, the view-frustum culling 
represents an implicit identification of intersecting view-frustums (i.e. retained view- 
frustums). That is, the retained view-frustums are those that intersect. 

27. The teachings of McAllister et al. and Davidson et al. are combinable because 
they are analogous art. Specifically, the teachings of both are directed toward the 
construction of 3D models of scenes from a collection of 2D images. Therefore, it would 
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have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed 
invention, to integrate the partitioning scheme of McAllister et al. into the method of 
Davidson et al. The motivation to do so would have been to and eliminate redundant 
views and imagery from the data set (McAllister et al. Section 4.5, paragraph 1 , 
sentence 3), thereby, improving rendering efficiency (McAllister et al. Section 4.6, last 
sentence) and storage overhead. Combining the teachings of McAllister et al. and 
Davidson et al., in this manner, yields a 3D modeling method that conforms substantially 
to that of claim 4. 

28. The following is in regard to Claim 5. As just shown, the teachings of McAllister et 
al. and Davidson et al. can be combined to yield a method that adequately satisfies the 
limitations of claim 4. Furthermore, the partitioning of the images of the sequence, 
according to the teachings of McAllister, is performed by removing, from the list 
associated with an image, the images which possess too few pixels corresponding to 
those of the current image (McAllister et al. Section 4.6, paragraph 2). Therefore, 
combining the teachings of McAllister et al. and Davidson et al., in the manner discussed 
above, yields a 3D modeling method that conforms substantially to that of claim 5. 

29. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Davidson et al., in view of La Roux et al. ("An Overview of Moving Object Segmentation 
in Videoimages", IEEE, 1991). 

30. The following is in regard to Claim 10. As shown above, Davidson et al. disclose 
a 3D model construction method that sufficiently conforms to the method of claim 1 . As 
mentioned above, Davidson et al. attempt to find pixels in the sequence of images that 
correspond to the same (or approximately the same) 3D point of the imaged scene or 
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object. The following problems associated with deriving stereo correspondences 
between images of moving objects are well known. Besides potentially inducing 
undesirable motion blur in the captured images, the movement of an object makes the 
derivation of correspondences between images of an image sequence extremely difficult 
or impossible. Though it would be apparent to one of ordinary skill in the art, the method 
of Davidson et al. does not provide a means to remedy this potential problem. 

31 . Le Roux, et al. describe several methods to differentiate and extract regions 
corresponding to moving objects in video images (Le Roux et al. Introduction). 

32. The teachings of Le Roux et al. and Davidson et al. are combinable because 
they are analogous art. In particular, the teachings of both Le Roux et al. and Davidson 
et al. are directed toward the processing of a sequence of images. Therefore, it would 
have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed 
invention, to use moving object segmentation, according to any of the various 
techniques discussed by Le Roux et al., to extract regions corresponding to moving 
objects from the collection of images used by the method of Davidson et al. to generate 
a 3D model. The motivation for doing so would have been to eliminate regions 
corresponding to moving objects from the collection of images. By eliminating these 
regions, correspondences, and hence a 3D model, can be derived for regions of these 
images that do not move. Combining the teachings of Le Roux et al. and Davidson et al., 
in this manner, yields a 3D modeling method that conforms substantially to that of claim 
10. 



Allowable Subject Matter 
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Objections, Allowable Subject Matter 

33. Claims 6-8 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

34. The following is a statement of reasons for the indication of allowable subject 
matter. 

35. The following is in regard to Claims 6-8. In and of themselves, the substantive 
limitations set forth in claims 6-8 introduce nothing over similar prior art methods. 
However, by virtue of their dependence on allowable claim 3 (see below), these claims, 
in turn, propose allowable subject matter. 

Allowable Subject Matter 

36. Claim 3 is allowed. 

37. The following is a statement of reasons for the indication of allowable subject 
matter. 

38. The following is in regard to Claim 3. As mentioned above with respect to claims 
1-2 , the method obtained by combining the teachings of Davidson et al. and 
Azarbayejani et al., involves assigning both a weight and resolution to pixels of the 
current image. Though one could arbitrarily designate these values as collectively 
constituting a relevance value, this does not follow directly or impliedly from the 
teachings of Davidson et al. or Azarbayejani et al. or, for that matter, any encountered 
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prior art methods. Despite this, it would be apparent, if one were to devise such a 
reference value, that in order for it to be meaningful - or, more specifically, for it to 
denote the relevance of pixels - within the context of the method obtained by combining 
the teachings of Davidson et al. and Azarbayejani et al. - the relevance value should be 
directly proportional to the resolution. This follows from the teachings of Davidson et al. 
(Davidson et al. Fig. 44a, steps S558-S560). Taking this into account, selected pixels, 
which according to Davidson et al. have highest 3D resolution (pixels with SHIFT < 10% 
of object size), would, in turn, have the highest relevance value among all image pixels. 
Providing an indication (e.g. a table) indicating which pixels are selected would be 
inherent to the process of selection. Such a collection can be considered a mask. 
Though these latter aspects of claim 3 follow implicitly from a meaningful notion of 
relevance value within the context of the method above, it cannot be said that these 
aspects would be readily apparent from the teachings of Davidson et al. or Azarbayejani 
et al. or any encountered prior art methods. In this manner, the subject matter of claim 3 
is unique and non-obvious with respect to the encountered prior art methods. 



Citation of Relevant Prior Art 

39. The prior art made of record and not relied upon is considered pertinent to 

applicant's disclosure: 

The following references relate to systems and methods that generate 3D models 
from image sequences. They have particular relevance to the various projection 
techniques disclosed by the Applicant. 

[1] U.S. Patent 6,731,283. Navab. Publication Date: May 2004. 
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[2] U.S. Patent 5,818,959. Webb et al. Publication Date: October 1998. 
The following references disclose systems and methods that generate 3D models 
from image sequences. 

[4] U.S. Patent 6,278,460. Myers et al. Publication Date: August 2001 

[5] Metric 3D Surface Reconstruction from Uncalibrated Image Sequences. Proc. 

SMILE Workshop (post-ECCV98), LNCS 1506, pp.1 38-1 53, Springer-Verlag, 

1998. Pollefeys et al. 

The method Pollefeys et al. calculates the fundamental matrix for each pair of 
views in the sequence of images. The derived matrices provide a projection of 
points from one view to the other of the pair. Pollefeys et al. also discuss a 
relevance (interest points) assigned to pixels, as well as pixel resolution (i.e. 
pixel width and height with respect to a 3D reference space). 
Other relevant prior art includes: 

[7] US. Patent 6, 124,864. Madden et al. Publication Date: September 2000. 
Madden et al. disclose an method that generates a 3D model from visual 
image streams. The method involves partitioning the images of the image 
sequence. 

[8] U.S. Patent 6,208,347. Migdal et al. Publication Date: March 2001. 

Migdal et al. disclose a method for 3D model construction that has a dynamic 
resolution capability, wherein the model is simplified by taking into 
consideration the resolution of 3D points constituting it. Migdal et al. also 
ascribe a relevance to points which incorporates their resolution. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kevin Siangchin whose telephone number is (703)305- 
7569. The examiner can normally be reached on 9:00am - 5:30pm, Monday - Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703)308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 

Kevin Siangchin 



Examiner 
Art Unit 2623 
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